Data Cleaning Process for mHealth Log Data to Inform Health Worker Performance.

Log data, captured during use of mobile health (mHealth) applications by health providers, can play an important role in informing nature of user engagement with the application. The log data can also be employed in understanding health provider work patterns and performance. However, given that these logs are raw data, they require robust cleaning and curation if accurate conclusions are to be derived from analyzing them. This paper describes a systematic data cleaning process for mHealth-derived logs based on Broeck's framework, which involves iterative screening, diagnosis, and treatment of the log data. For this study, log data from the demonstrative mUzima mHealth application are used. The employed data cleaning process uncovered data inconsistencies, duplicate logs, missing data within logs that required imputation, among other issues. After the data cleaning process, only 39,229 log records out of the initial 91,432 usage logs (42.9%) could be included in the final dataset suitable for analyses of health provider work patterns. This work highlights the significance of having a systematic data cleaning approach for log data to derive useful information on health provider work patterns and performance.

Factors Associated with the Willingness to Perform a Physical Employment Standard in Probation Officers.

PURPOSE: This study examined worker characteristics and behaviors and the perceived importance of job-related tasks associated with the willingness to perform a proposed probation officer physical ability test (PROPAT). METHODS: North Carolina probation officers (N = 1213, 46.2% female, 39.8 ± 10.1 yr, 30.7 ± 6.6 kg·m-2) completed a survey including demographics, health history, and job-related tasks. A multivariable logistic regression model estimated the odds of being willing to perform the PROPAT. Adjusted odds ratios (ORadjusted) with 95% confidence intervals (CI) were calculated, with those excluding 1.00 deemed statistically significant. RESULTS: The majority (72%) of probation officers were willing to perform the PROPAT. Being male (compared with female) (ORadjusted = 1.50, 95% CI = 1.12-2.02) and having moderate (ORadjusted = 1.51, 95% CI = 1.10-2.08) and high physical activity status (ORadjusted = 2.89, 95% CI = 1.97-4.28) (compared with low) increased the odds of being willing to perform the PROPAT. Additionally, reporting a greater importance of tasks, including running to pursue a suspect (TASKRUN, 1-unit increase, ORadjusted = 1.25, 95% CI = 1.10-1.43) and dragging an unresisting person (TASKDRAG, 1-unit increase, ORadjusted = 1.16, 95% CI = 1.01-1.32), increased the odds of being willing to perform the PROPAT. Increasing age (1-yr increase, ORadjusted = 0.96, 95% CI = 0.94-0.98), class II (ORadjusted = 0.45, 95% CI = 0.28-0.71) and class III obesity (compared with normal weight, ORadjusted = 0.55, 95% CI = 0.32-0.93), and a previous musculoskeletal injury (ORadjusted = 0.57, 95% CI = 0.35-0.93) all decreased the odds of being willing to perform the PROPAT. CONCLUSIONS: These results suggest being older, previously injured, obese (body mass index ≥ 35 kg·m-2), and less active is associated with being less willing to participate in the PROPAT, whereas males and reporting a higher importance of the TASKRUN and TASKDRAG activities is associated with being more willing to participate in the PROPAT. Departments can use these findings to identify feasible strategies (e.g., education and physical training) to improve the implementation of physical employment standards.

Teaching and assessing intra-operative consultations in competency-based medical education: development of a workplace-based assessment instrument.

Competency-based medical education (CBME) is being implemented worldwide. In CMBE, residency training is designed around competencies required for unsupervised practice and use entrustable professional activities (EPAs) as workplace "units of assessment". Well-designed workplace-based assessment (WBA) tools are required to document competence of trainees in authentic clinical environments. In this study, we developed a WBA instrument to assess residents' performance of intra-operative pathology consultations and conducted a validity investigation. The entrustment-aligned pathology assessment instrument for intra-operative consultations (EPA-IC) was developed through a national iterative consultation and used clinical supervisors to assess residents' performance at an anatomical pathology program. Psychometric analyses and focus groups were conducted to explore the sources of evidence using modern validity theory: content, response process, internal structure, relations to other variables, and consequences of assessment. The content was considered appropriate, the assessment was feasible and acceptable by residents and supervisors, and it had a positive educational impact by improving performance of intra-operative consultations and feedback to learners. The results had low reliability, which seemed to be related to assessment biases, and supervisors were reluctant to fully entrust trainees due to cultural issues. With CBME implementation, new workplace-based assessment tools are needed in pathology. In this study, we showcased the development of the first instrument for assessing resident's performance of a prototypical entrustable professional activity in pathology using modern education principles and validity theory.

Prevalence, Indirect Costs, and Risk Factors for Work Disability in Patients with Crohn's Disease at a Tertiary Care Center in Rio de Janeiro.

BACKGROUND AND AIMS: Crohn's disease (CD) can lead to work disability with social and economic impacts worldwide. In Brazil, where its prevalence is increasing, we assessed the indirect costs, prevalence, and risk factors for work disability in the state of Rio de Janeiro and in a tertiary care referral center of the state. METHODS: Data were retrieved from the database of the Single System of Social Security Benefits Information, with a cross-check for aid pension and disability retirement. A subanalysis was performed with CD patients followed up at the tertiary care referral center using a prospective CD database, including clinical variables assessed as possible risk factors for work disability. RESULTS: From 2010 to 2018, the estimated prevalence of CD was 26.05 per 100,000 inhabitants, while the associated work disability was 16.6%, with indirect costs of US$ 8,562,195.86. Permanent disability occurred more frequently in those aged 40 to 49 years. In the referral center, the prevalence of work disability was 16.7%, with a mean interval of 3 years between diagnosis and the first benefit. Risk factors for absence from work were predominantly abdominal surgery, anovaginal fistulas, disease duration, and the A2 profile of the Montreal classification. CONCLUSIONS: In Rio de Janeiro, work disability affects one-sixth of CD patients, and risk factors are associated with disease duration and complications. In the context of increasing prevalence, as this disability compromises young patients after a relatively short period of disease, the socioeconomic burden of CD is expected to increase in the future.

Quality of dictated feedback associated with SIMPL operative assessments of pediatric surgical trainees.

BACKGROUND: SIMPL is a workplace-based operative performance assessment tool which allows for dictated feedback (DF). To better understand the value of DF, we sought to characterize the type and quality of DF generated during SIMPL evaluations. METHODS: Thematic analysis of DF from SIMPL assessments between June 2017 and December 2018 at a single pediatric surgery fellowship program was performed. Comments were categorized as specific, encouraging or corrective. Categories were combined to determine DF quality as effective, mediocre or ineffective. RESULTS: Of 781 SIMPL assessments (21 faculty, 5 trainees), 451 (57%) had DF. Most comments were encouraging (93%) and specific (65%). Only 21% were corrective, 17% had entrustment features, and 8% had an explicit learning plan. Feedback quality was deemed mediocre (45%), ineffective (33%) and effective (21%). CONCLUSION: SIMPL dictated feedback was mostly encouraging and specific. To improve quality, feedback should incorporate learning plans as well as corrective and entrustment features.

Impaired Quality of Working Life in Inflammatory Bowel Disease Patients.

BACKGROUND: Work-related aspects are important determinants of health for inflammatory bowel disease (IBD) patients. AIMS: We aimed to describe quality of working life (QWL) in IBD patients and to assess variables that are associated with QWL. METHODS: Employed IBD patients of two tertiary and two secondary referral hospitals were included. QWL (range 0-100) was measured using the Quality of Working Life Questionnaire (QWLQ). Work productivity (WP), fatigue, and health-related quality of life (HRQL) were assessed using the Work Productivity and Activity Impairment questionnaire, Multidimensional Fatigue Inventory, and Short Inflammatory Bowel Disease Questionnaire, respectively. Active disease was defined as a score > 4 for the patient-reported Harvey-Bradshaw index in Crohn's disease (CD) or Simple Clinical Colitis Activity Index in ulcerative colitis patients. RESULTS: In total, 510 IBD patients were included (59% female, 53% CD, mean age 43 (SD 12) years). The mean QWLQ score was 78 (SD 11). The lowest subscore (54 (SD 26)) was observed for "problems due to the health situation": 63% reported fatigue-related problems at work, 48% agreed being hampered at work, 46% had limited confidence in their body, and 48% felt insecure about the future due to their health situation. Intermediate/strong associations were found between QWL and fatigue (r = - 0.543, p < 0.001), HRQL (r = 0.527, p < 0.001), WP loss (r = - 0.453, p < 0.001) and disease activity (r = - 0.331, p < 0.001). Independent predictors of impaired QWL in hierarchical regression analyses were fatigue (B = - 0.204, p < 0.001), WP loss (B = - 0.070, p < 0.001), and impaired HRQL (B = 0.248, p = 0.001). CONCLUSIONS: IBD-related problems at work negatively influence QWL. Fatigue, reduced HRQL, and WP loss were independent predictors of impaired QWL in IBD.

Nursing Student Medication Administration Performance: A Longitudinal Assessment.

BACKGROUND: Nurses must be adequately prepared for safe medication administration. PURPOSE: The specific aims of the study were to compare first- and final-semester nursing students' medication administration performance, describe graduating nursing student performance, assess the long-term effect of an Individual Simulation-Based Experience (ISBE), and determine if nursing assistant experience impacted performance. METHODS: The study was a longitudinal, descriptive, experimental, 2-group pretest-posttest design. The Medication Administration Safety Assessment Tool was used to assess nursing student (n = 68) medication administration performance at 3 points in time: after learning the skill, after participation in an ISBE or traditional practice session, and prior to graduation. RESULTS: Only one-third of senior nursing students accurately demonstrated medication administration. There was no significant difference between first- and last-semester performance. CONCLUSION: Findings suggest nursing students are not prepared for safe medication administration upon graduation. The reality of practice opportunities to perform this skill throughout a nursing program should be examined.

A novel faculty development tool for writing a letter of recommendation.

OBJECTIVE: Based on a national survey of program directors we developed a letter of recommendation (LOR) scoring rubric (SR) to assess LORs submitted to a pediatric residency program. The objective was to use the SR to analyze: the consistency of LOR ratings across raters and LOR components that contributed to impression of the LOR and candidate. METHODS: We graded 30 LORs submitted to a pediatric residency program that were evenly distributed based on final rank by our program. The SR contained 3 sections (letter features, phrases, and applicant abilities) and 2 questions about the quality of the LOR (LORQ) and impression of the candidate (IC) after reading the LOR on a 5-point Likert scale. Inter-rater reliability was calculated with intraclass correlation coefficients (ICC(2,1)). Pearson (r) correlations and stepwise multivariate linear regression modeling predicted LORQ and IC. Mean scores of phrases, features, and applicant abilities were analyzed with ANOVA and Bonferroni correction. RESULTS: Phrases (ICC(2,1) = 0.82, p<0.001)) and features (ICC(2,1) = 0.60, p<0.001)) were rated consistently, while applicant abilities were not (ICC(2,1) = 0.28, p<0.001)). For features, LORQ (R2 = 0.75, p<0.001) and IC (R2 = 0.58, p<0.001) were best predicated by: writing about candidates' abilities, strength of recommendation, and depth of interaction with the applicant. For abilities, LORQ (R2 = 0.47, p<0.001) and IC (R2 = 0.51, p<0.001) were best predicted by: clinical reasoning, leadership, and communication skills (0.2). There were significant differences for phrases and features (p<0.05). CONCLUSIONS: The SR was consistent across raters and correlates with impression of LORQ and IC. This rubric has potential as a faculty development tool for writing LORS.

Evaluating Nurse Practitioner Student Competencies: Application of Entrustable Professional Activities.

BACKGROUND: A knowledge gap exists regarding the best methods for assessment of nurse practitioner (NP) student clinical competence. Entrustable professional activities (EPAs) translate competencies into practice and provide a potential framework for NP clinical competency assessment. This study piloted the use of an EPA-based method of assessing NP student clinical competence via simulated clinical experiences. METHOD: Six EPAs were mapped to NP core competencies. Faculty designed clinical scenarios to assess student clinical performance using EPA-based rubrics. Online family nurse practitioner students rotated through simulated clinical scenarios during an on-campus residency. Faculty assigned a utility score to each EPA activity description. RESULTS: Student EPA scores paralleled time-based program requirements. Faculty found over 70% of EPA activity descriptions to be useful or requiring minor editing in assessing students. CONCLUSION: This study is an important step toward the creation of a standardized, competency-based process for NP clinical performance assessment. [J Nurs Educ. 2020;59(12):714-720.].

GP appraisal: an evaluation of generational differences on the utility of GP appraisal.

BACKGROUND: A number of studies have previously been published on the benefits of GP appraisal. Ours is the first study that compares the views of doctors registered before and after the introduction of appraisals on various core elements of the appraisal process. AIM: This study aimed to explore potential differences, between GPs qualifying before and those after the introduction of NHS appraisals, on the utility of the appraisal process. Additionally, to discover the perceived impact of appraisals on interaction with colleagues and patients. DESIGN AND SETTING: Suffolk appraisers were recruited to distribute a paper questionnaire, in two sections, for each appraisal undertaken over a 12-month period. The first part of the questionnaire related to that specific appraisal. The second part, seven questions using Likert scales and free-text comments, asked about the appraisal process in general. The feedback from the second part forms the data for this study. RESULTS: Overall, doctors tended to agree that appraisals had a beneficial impact in the core areas investigated. However, there was a significant difference between generations: those registered before 1998 were less likely to find NHS appraisals beneficial. Both groups reported that preparation took up too much time. Opinions regarding the impact of appraisals on interaction were evenly divided amongst appraisees, with the older cohort more likely to report negatively. CONCLUSIONS: There is a generational difference amongst GP in the perceived utility of NHS appraisal. Current plans to redesign the process to make appraisals less onerous will be welcomed by both groups.

Measuring achievement striving via a situational judgment test: the value of additional context / Medición del esfuerzo hacia el logro mediante de un test de juicio situacional: el valor del contexto adicional

The study extends personality and situational judgment test (SJT) research by using an SJT to measure achievement striving in a contextualized manner. Employed students responded to the achievement striving SJT, traditional personality scales, and workplace performance measures. The SJT was internally consistent, items loaded on a single factor, and scores converged with other measures of achievement striving. The SJT provided incremental criterion-related validity for the performance criteria beyond less-contextualized achievement striving measures. Findings suggest that achievement-related work scenarios may provide additional criterion-relevant information not captured by measures that are less contextualized

Este estudio extiende la investigación sobre los tests de juicio situacional (TJS) y la personalidad, usando un TJS para medir el esfuerzo hacia el logro de una manera contextualizada. Estudiantes con empleo respondieron el TJS de esfuerzo hacia el logro, escalas típicas de personalidad y medidas de desempeño en el trabajo. El TJS fue internamente consistente, los ítems cargaron en un único factor y las puntuaciones convergieron con otras medidas de esfuerzo hacia el logro. El TJS añadió validez relacionada con el criterio, para los criterios de desempeño, a la validez de las medidas menos contextualizadas de esfuerzo hacia el logro. Los hallazgos sugieren que escenarios de trabajo relacionados con el logro pueden proporcionar información adicional relevante para el criterio no capturada por medidas menos contextualizadas

Performance of nurses in the bed management service of a teaching hospital.

OBJECTIVES: to analyze the performance of nurses in the Bed Management service of a public teaching hospital. METHODS: descriptive, qualitative research inspired by Dubois theoretical model. 32 participants from the bed management unit and care units of a public hospital were interviewed from February to May 2018. Observations and data records from the Bed Management service complemented data collection. Theme/Category-Based Content Analysis was used. RESULTS: the performance of nurses was described in three analysis matrices, represented by the categories: acquisition, implementation and maintenance of nursing resources; transformation of nursing resources into nursing service and results of the work of the nurse in Bed Management. Final Considerations: understanding the performance of nurses in bed management contributes to the scenario of new practices and different roles that favor visibility and strengthening of the nurse's identity.

Performance management in primary healthcare: Nurses' experiences.

BACKGROUND: The use of the performance management (PM) system is highly contested by public servants in South Africa, although its value as essential to the appraisal and management of staff is undeniable. OBJECTIVES: The aim of this study was to explore nurses' perceptions and experiences of the PM system at primary healthcare (PHC) facilities in relation to the current health system's reforms. METHOD: An exploratory, descriptive and qualitative design was utilised. Participants were selected through purposive sampling. A semi-structured interview tool was used to collect data from 18 nurses in four sub-districts of Dr. Kenneth Kaunda district in the North West province. Data were analysed through thematic analysis. RESULTS: The findings of this study confirmed that PM is implemented to some extent. However, various loopholes in its implementation threaten the accuracy and transparency of the system and leave it vulnerable to perceived organisational injustice and unfairness, with the objectivity of the system questioned. The limitations of the current PM system revealed by this study include (1) the lack of alignment with current health system reforms towards comprehensive and integrated care that demands person-centred care; (2) the system's usefulness for career progression, performance improvement and rewarding exceptional performance. CONCLUSION: Performance management is inadequately applied in PHC facilities at district level and needs to be realigned to include the appraisal of key attributes required for the current health system's reforms towards comprehensive and integrated care, including the provision of person-centred care, which is central for responding adequately to South Africa's changing disease profile towards multi-morbidity.